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ABSTRACT: Image fusion is the technique of merging or combining two or more images to give a 
single image with enhanced information content. Most of the standalone image fusion techniques have 
their own advantages as well as disadvantages. So it is always desirable to pursue a combination of two 
or more techniques to give better results. The fusion technique employed is a combination of the existing 
Pyramid based fusion and Edge preserving smoothing filter based fusion rules. First an edge preserving 
smoothing filter such as Bilateral filter or Guided filter is used to obtain base and detail layers of the 
input images. The base and detail layers are separately fused using Pyramid fusion rule. The fusion of 
base layers are controlled by a weighting factor which takes into consideration the exposure, contrast, 
and saturation measures of the input images. The fused base and detail layers are combined to give the 
final image. The work also gives a comparative study of the performance of the two common edge 
preserving smoothing filters - Bilateral filter and Guided filter. Commonly used Image quality metrics 
such as PSNR and SSIM are employed for analyzing the experimental results. An average PSNR value of 
62.68 is obtained with Guided filter and 60.03 with Bilateral filter. 
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I. Introduction 

With the rapid advancement in science and technology the digital world is flooded with a large variety of images 
which are of different resolutions and qualities. To analyze a large no of images is a tedious task where the 
technique of image fusion can be effectively employed. Image fusion in simple terms is the fusion or merging of 
two or more images. By fusing two images of the same object we aim to obtain an output image with enhanced 
information content. An efficient image fusion method should retain the desired features of the component 
images while rejecting the unwanted information. 

There are three main constraints for Image fusion 
o The method should be resistant to misregistration errors, 
o The fused image should preserve all relevant information from the input images 
o The fused image should not contain artifacts which can lead to a wrong diagnosis. 

Image fusion aims at data integration in order to obtain a single image with more information than can 
be derived from each of the images alone. This data integration has a major advantage that the volume of data to 
be analyzed gets reduced as the single fused image contains all the relevant information contained in two or 
more source images. 

Image fusion can be done on a variety of images such as multi view, multi focus, multi temporal, multi 
exposure, multi spectral multi modal images etc. However we should be careful while choosing source images as 
they should be properly registered i.e., they should be spatially aligned. 

Image fusion finds a wide range of applications in computer vision, automatic object detection, 
robotics, remote sensing, medical imaging, etc. 



II. Common Image Fusion Techniques 

The common Image fusion techniques can be broadly classified into- Spatial domain, Transform domain and 
Multiple algorithm fusion. 

2.1 Spatial domain image fusion 

In spatial domain techniques, we directly deal with the image pixels. The source images are fused using 
local spatial features such as gradient, spatial frequency, and local standard deviation. Commonly used spatial 
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domain methods are maximum or minimum, Brovery method [1], Principal component analysis (PCA)[2], High 
pass filtering, Intensity Hue Saturation (IHS) [3] [4] etc. The spatial domain approaches has a disadvantage that 
they produce spatial distortion in the fused image. 

2.2 Transform domain image fusion 

In the transform domain method the images are transformed into the transform domain and the required 
fusion operations are done and the obtained fused image is inverse transformed to obtain the output image. 
Popular transform domain techniques are DWT [5], Laplacian pyramid [6] [7], Curvelet transform[8] etc. 

2.3 Multiple algorithm image fusion 

Both the spatial and the transform domain techniques have its own advantages and disadvantages so it 
is desirable to use a combination of spatial and transform domain methods to obtain optimum results. For 
example combination of DWT with PCA techniques gives superior fusion performance. Most of the research 
works are being concentrated on composite image fusion techniques which rely on combination of two or more 
image fusion techniques to give enhanced results 

III. Proposed Method 

The fusion technique used in this paper employs a combination of edge preserving smoothing filter (bilateral 
filter or guided filter) and a multi resolution technique such as Laplacian pyramid. The Fig. 1 shows the design 
flow of the proposed method which is explained in detail. 




Figure 1 : Framework of the proposed method 



3.1 Load images 

The source images for image fusion can be of multi view, multi focus, multi exposure, multi temporal, 
and multi modal types provided they are of the same scene. 

3.2 Edge preserving smoothing filter 

The edge preserving smoothing filter enhances fine details such as edges and boundaries in the fused 
image while smoothing away textures. The proposed method uses two of the most common filters i.e. Bilateral 
filter [9] and Guided filter [10]. 

Bilateral filtering is a local, nonlinear and non-iterative technique which combines a classical low-pass 
filter with an edge-stopping function that attenuates the filter kernel when the intensity difference between pixels 
is large. As both gray level similarities and geometric closeness of the neighboring pixels are considered, the 
weights of the filter depend not only on Euclidian distance but also on the distance in gray/color space. 
Cross/Joint bilateral filter [9] is a bilateral filter which is effective for detecting and reducing large artifacts such 
as reflections using gradient projections it uses one image for finding the kernel and other to filter, and vice 
versa. The figure given below illustrates the operation of an edge preserving smoothing filter. 



I IJMER I ISSN: 2249-6645 I 



www.ijmer.com 



I Vol. 5 I Iss.4l Apr. 2015 I 281 



A Comparative Study of Edge Preserving Smoothing Filters in Pyramid Based Image Fusion 



A (Reference image) 
B (Input image) 






( " A 

* Edge preserving 
smoothing filter 

V ) 




Figure 2: Edge preserving filter operation 



B b (Base layer) 



^ B d (Detail layer) 



The source images A and B are the inputs. B b is the filter out put with image B as input. Image A is reference 
image used for creating filter kernel. B b is the base layer and B D is the corresponding detail layer. 

The Bilateral filter output of image B at a pixel location p is calculated as: 

B b(P) = ^ ^ ( | IP - <? 1 1) G crr (MCp) “ A iq) l)S(<?) (1) 

where G ar (|A(p) — A(q) |) is a gray level similarity function [9] and W is the normalization constant. 



Guided filter is also an edge preserving smoothing filter similar to the bilateral filter with some 
advantages - faster than bilateral filter and avoids staircasing and gradient reversal artifacts. The guided filter 
performs edge-preserving smoothing on an image, using the content of a guidance image. The guidance image 
can be selected according to the requirement. Guided image filtering is a neighborhood operation, like bilateral 
filtering, but takes into account the statistics of a region in the corresponding spatial neighborhood in the 
guidance image when calculating the value of the output pixel. The structures in the guidance image will impact 
the filtered image in effect, imprinting these structures on the original image. This effect is called structure 
transference. 

The working of guided filter [10] is similar to bilateral filter. Here I is the guidance image p is the input 
image and B g is the guided filtered output of I in a window &> fe centered at the pixel k: 

B a< = a *h + K ’ v , e "fr (2) 

where a k and b k are linear coefficients obtained from the input image p. 

The output of the edge preserving filter computed from (1) and (2) is the smoothened image which 
comprises the base layer represented as B k for the k th input image I k . The detail layer is obtained by taking the 
difference between the original image and the base layer, which is defined as: 

®k = h ~ Bk ( 3 ) 



3.3 Computation of Gaussian and Laplacian Pyramid 

The Pyramid representation [6] [7] is a type of multi-scale signal representation in which a signal or an 
image is subjected to repeated smoothing and subsampling. A Gaussian pyramid is an image processing 
technique used in texture synthesis of the images. The technique involves creating a series of images which are 
weighted down using a Gaussian blur and scaled down 

A pyramid is created by low pass-filtering or averaging an image G 0 with a compact two-dimensional 
filter [11]. The filtered image is then subsampled by removing every other pixel and every other row to obtain a 

reduced image Gi.This process is repeated to form a Gaussian pyramid G 0 ,Gi,G 2 ,G 3 , G d 

Expanding G\ to the same size as G 0 and subtracting yields the band-passed image L 0 . A Laplacian 

pyramid Lo,Li,L 2 ,L 3 , Ld-i, can be built containing band-passed images which are of decreasing size and 

spatial frequency: 

L j = G l - G l+1 l = 1 , 2 , 3 , ... . d - 1 , ( 4 ) 



where each Laplacian level contains the edge details of the input images. 

3.4 Base layer fusion 

The fused base layer B f obtained as the weighted sum of the individual base layers of the input images. 
The pyramid approach [7] generates Laplacian pyramid of the base layers L{B k }and Gaussian pyramid [11] of 
the weight map functions G{W k } which is obtained by the three quality factors - saturation S k , contrast C k and 
exposure E k . The weight map is given by the product of the three quality factors: 
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Wit= S k ■ C k E k (5) 

Laplacian pyramid of the baser layers multiplied with corresponding Gaussian pyramid of weight map and 
summing over K gives the modified Laplacian pyramid L 1 : 

N 

L l = ( 6 ) 

Jt = l 

Where l (0< l < d) is the number of levels in the pyramid and K (7 < K <N) is the number of input images. 

The fused base layer B f which contains well exposed pixels is reconstructed by expanding each level 
and then summing all the levels of the Laplacian pyramid: 

gE 

B f = (7) 

2 = Q 



3.5 Detail layer fusion 

The detail layers computed in (3) from all input images are linearly combined to produce fused detail 
layer D f which gives combined texture information: 

n _ 

f~ N 

where y is the parameter to control amplification of texture details and 
reduce noise and artifacts near edges while achieving detail enhancement 

3.6 Two Scale Reconstruction 

The final fused image F is obtained by simply adding or combining the fused base layer B f and the 
fused detail layer D f 

F= B f + D f (9) 



( 8 ) 

f k () is a non linear function [11] to 



IV. Experimental Results And Analysis 

The simulation of the proposed method was performed using MATLAB 8.1 (R2013a). The input images can 
be of a variety of categories - multi focus, multi temporal, multi exposure or multi modal which are fused using 
the proposed method to give optimum results. We are comparing the performance of the proposed method on 
two different perspectives, based on the edge preserving smoothing filter used. Thus we are comparing the 
performance of the two common edge persevering filters, Bilateral filter versus Guided filter. 




Figure3: Input images (Multi focus) Figure 4: Output Image (Fused Image) 

We use PSNR (Peak Signal to Noise Ratio) to evaluate quality of the fused image. PSNR is n approximation to 
human perception of reconstruction quality. PSNR is most easily defined via the mean squared error (MSE) : 

m-ln-l 

mse= ^VL (1 ° 

i=0 j = 0 
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PSNR = 10 ■ lo 3l0 ( ' MAX ‘/ MSE ) (11) 

where H iJ) and P(i,f) are the original image and fused image respectively and MAX l stands for the 
maximum pixel value of the image. PSNR values falling below 30 dB indicate fairly a low quality. Therefore 
high quality images must have a value of 40 dB or more. 

Structural similarity (SSIM) [12] is the second quality metric employed. This metric is employed for measuring 
the similarity between two images. The principle behind SSIM is that spatially close pixels have strong inter- 
dependencies. SSIM defines image degradation as perceived change in structural information. SSIM between 
two images of same size can be calculated as: 

+ Ci)(2tr /f + c 2.) (12) 

{f ' F] Qif + fij + cOiof + aj + c 2 ) 

Where and p F are the average and <sf and <J F are the variance of the Input and output images respectively 
and <t if is the covariance of the input and output images. cl and c2 are variables to stabilize the division with 
weak denominator. 



Table 1: Quality Metrics of the Fused Image 



INPUT 


PSNR 


SSIM 


IMAGE 


BILATERAL 


GUIDED 


BILATERAL 


GUIDED 




FILTER 


FILTER 


FILTER 


FILTER 


Multi focus 


64.1355 


70.2197 


0.9968 


0.9989 


Multi temporal 


60.3433 


62.1286 


0.9905 


0.9941 


Multi exposure 


56.0057 


55.9049 


0.9774 


0.9767 


Multi modal 


59.675 


62.4711 


0.9892 


0.9945 



PSNR comparison 




Source Images 
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SSIM comparison 




Speed comparison 




Source Images 

Figure 5 : Performance comparison curves of the proposed method 

4.1 Performance Analysis of the Proposed method. 

The proposed method employs an edge preserving smoothing filter along with pyramid fusion to 
obtain excellent results for the fusion of a variety of images as shown by the performance curves. The 
experiments were conducted on four different sets of input images- multi focus, multi temporal, multi exposure 
and multimodal images. The two curves correspond to two different edge preserving smoothing filters 
employed. From the performance curves it is quite evident that Guided filter when employed as the edge 
preserving smoothing filter, gives superior results than Bilateral filter, except in the case of multi exposure 
images. But taking into consideration the speed of operation, guided filter surpasses bilateral filter in all the 
four cases. The proposed method gives best fusion results with multi focus images as the input images. 

V. Conclusion 

Composite image fusion techniques are now replacing standalone image fusion techniques. This is also an active 
research area. The fusion rule combines the advantages of Edge preserving smoothing filter and Pyramid based 
fusion to give optimum results. Four different sets of input images- multi focus, multi temporal, multi exposure 
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and multi modal were considered. The Pyramid based fusion using Edge preserving filter gives good results with 

the common quality metrics such as PSNR and SSIM. From the comparative analysis of the performance of the 

Edge preserving smoothing filters in the proposed method, employing Guided filter as Edge preserving 

smoothing filter results in higher quality fused images than Bilateral filter. 
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